Skip to content

test: update test_request_deduplication_edge_cases#884

Merged
vdusek merged 3 commits into
apify:masterfrom
Mantisus:update-test_request_deduplication_edge_cases
May 29, 2026
Merged

test: update test_request_deduplication_edge_cases#884
vdusek merged 3 commits into
apify:masterfrom
Mantisus:update-test_request_deduplication_edge_cases

Conversation

@Mantisus
Copy link
Copy Markdown
Collaborator

@Mantisus Mantisus commented May 3, 2026

Closes: #786

Re-enables test_request_deduplication_edge_cases in shared mode (previously skipped as flaky).

The flakiness came from add_request occasionally returning None transiently due to platform-side propagation in shared request-queue mode. Instead of a fixed retry loop with no delay between attempts, the test now uses the poll_until_condition helper, which polls add_request with backoff until it returns a result (or times out). This gives transient None results time to clear and is consistent with how other integration tests handle eventually-consistent API state.

@Mantisus Mantisus requested a review from vdusek May 3, 2026 23:55
@Mantisus Mantisus self-assigned this May 3, 2026
Copy link
Copy Markdown
Contributor

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but I would rather see the root cause investigation and actually trace where the empty processedRequests comes from, rather than retry in test workaround 🙂.

@janbuchar
Copy link
Copy Markdown
Contributor

Thanks, but I would rather see the root cause investigation and actually trace where the empty processedRequests comes from, rather than retry in test workaround 🙂.

It's a frustrating answer, but due to the nature of the Request Queue, this sometimes just... happens. It is an established fact that the API leaks implementation details. Maybe we should make sure we plug that leak at the SDK level at least, though.

@vdusek
Copy link
Copy Markdown
Contributor

vdusek commented May 4, 2026

It's a frustrating answer, but due to the nature of the Request Queue, this sometimes just... happens. It is an established fact that the API leaks implementation details. Maybe we should make sure we plug that leak at the SDK level at least, though.

Yeah, in that case, we should try to address this in the ApifyStorageClient

Copy link
Copy Markdown
Contributor

@vdusek vdusek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just utilized the poll_until_condition, otherwise LGTM.

@vdusek vdusek force-pushed the update-test_request_deduplication_edge_cases branch from 96e5038 to c90b05e Compare May 29, 2026 15:38
vdusek added 2 commits May 29, 2026 17:40
Replace the no-delay retry loop with the poll_until_condition helper so
transient platform-side None results have time to clear between attempts.
@vdusek vdusek force-pushed the update-test_request_deduplication_edge_cases branch from c90b05e to f5835aa Compare May 29, 2026 15:40
@vdusek vdusek merged commit d290e40 into apify:master May 29, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test test_request_deduplication_edge_cases[shared] is flaky

4 participants